test(cloudflare): Skip retries for expected failures - #22991
Conversation
Avoid retrying intentional worker failures, which only repeats exceptions and slows tests. Add Durable Object and scheduled-handler error coverage. Co-authored-by: Cursor <cursoragent@cursor.com> Co-Authored-By: Claude <noreply@anthropic.com>
|
|
||
| try { | ||
| const res = await fetchWithRetry(url, { headers, method, body }); | ||
| const res = await fetchWithRetry(url, { headers, method, body }, expectError ? { maxRetries: 1 } : {}); |
There was a problem hiding this comment.
Bug: When expectError: true and maxRetries: 1 are used, a connection error during startup is silently suppressed, causing tests to hang and time out.
Severity: MEDIUM
Suggested Fix
When expectError: true, the test runner should still retry on transient connection errors (e.g., ECONNREFUSED) but should stop retrying once it successfully receives any response from the worker, including the expected error response. This will prevent connection failures from being silently swallowed while correctly handling expected application-level errors. Alternatively, only use maxRetries: 1 after verifying the worker is truly accepting connections.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: dev-packages/cloudflare-integration-tests/runner.ts#L441
Potential issue: When a test is configured with `expectError: true` and `maxRetries: 1`,
a race condition during wrangler startup can cause a connection error. The `makeRequest`
function's `catch` block silently suppresses this specific type of error, preventing the
request from ever reaching the worker. Because the worker is never hit, the Sentry
envelope the test is waiting for is never generated. This causes the test to hang until
it eventually fails due to a timeout, creating a flaky test condition that obscures the
root cause of the failure.
Also affects:
dev-packages/cloudflare-integration-tests/runner.ts:84~88
Did we get this right? 👍 / 👎 to inform future reviews.
size-limit report 📦
|
This adds the
expectErrorflag, for tests which are expected to fail, e.g.tracing/headershad a runtime of 5seconds which trimmed it down to 500ms with that flag (as it retried 3 times - even if it shouldn't had to)It also seemed that we didn't cover unhandled DO or schedule errors in the integration suite, so I added them here too